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(54) Abstract Title 

Television remote control with display 

(57) A remote control 110 for a multimedia TV receiver 
(30, fig.1) comprises a display screen 720. such as an LCD 
display device, user-operable control 700 (e.g. keyboard) 
and a processor 710 and wireless bi-directional 
communications interface 730. A system controller (100) 
receives requests* from the remote control device, for 
Information relating to the video signal currently being 
displayed on the main display of the TV (30). This 
information is then communicated to the remote control 
and displayed on the display screen 720 without being 
displayed on the main display screen of the TV (30). The 
information in question may be an electronic program 
guide, teletext information, subtitling or related material 
derived from the Intemet. The electronic program guide 
contains keywords that may be used as the basis of an 
Internet search (by the system controller (100)) for 
information related to the currently received TV program. 
This information Is then transmitted to the remote control 
for display. Alternatively the remote control may be used 
to request replay of images from the main display (30) on 
the remote control display 720, or to browse the Internet. 
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At least one drawing origmally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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2343073 

1 

TELEVISION RFCFJVTNG AP PARATUS WITH RKMOTF. rONTROT 

This invention relates to television receiving apparatus with remote control 
devices. 

S In a conventional home entertainment system a remote control device is used to 

control the operations of the home entertainment equipment - for example, a video 
recorder, a television, an audio system and so on. 

Originally a respective individual remote control device would have been 
required to service each of these pieces of equipment. More recently, combined 
10 remote control devices have become available which are capable of controlling severed 
independent or interconnected pieces of equipment. 

This invention provides television receiving apparatus comprising: 

a signal receiver; 

a system controller; 

15 a main display screen operable to display a video signal from the signal 

receiver; and 

a remote control operable to communicate with the system controller, the 
remote control having a display device and a user-operable control; 

in which the remote control comprises means, responsive to user operation of 
20 the user-operable control, for requesting transfer from the system controller of 
information relating to a video signal currently displayed on the main display screen, 
the information being communicated from the system controller to the remote control 
for display on the display device of the remote control. 

The invention recognises that a major limitation of current remote control 
25 devices for television receivers is that the flow of information is primarily one way. 
The invention links the functionality of the remote control and the television receiving 
apparatus much further by providing the remote control with a display device and 
providing an arrangement whereby information relating to a cxmrently viewed 
television programme can be viewed on the remote control's display device. 
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This interactivity of the remote control device can have many advantages. For 
example, if the user wishes to check some teletext information he can do so without 
disrupting viewing for other users. In preferred embodiments, the user can request a 
replay of a recentiy viewed section of the programme to be displayed locally on the 
5 remote control's display, again with no disruption to other viewers. 

Embodiments of the invention will now be described, by way of example only, 
with reference to the accompanying drawings in which: 

Figure 1 is a schematic diagram of a home entertainment system; 

Figure 2 is a schematic diagram of a set-top-box; 
10 Figxire 3 is a flow chart illustrating the formation of a user action log; 

Figure 4 is a flow chart illustrating the formation of an adaptive programme 

guide; 

Figure 5 schematically illustrates the parsing of electronic programme guide 
data for inclusion in the user action log; 
15 Figures 6 to 8 schematically illustrate a distribution of data stored in the user 

action log; 

Figure 9 is a flowchart illustrating a recording operation; 
Figure 10 is a schematic diagram illustrating user privileges when a single user 
is operating the system of Figure 1 ; 
20 Figures 11 and 12 illustrate user privileges when two or more users are 

operating the system of Figure 1; 

Figure 13 schematically illustrates the derivation of highlight and/or preview 
programme clips; 

Figure 14 schematically illustrates a remote control device; 
25 Figure 15 is a flow chart illustrating the display of related material on the 

remote control device; and 

Figure 16 is a flow chart illustrating the display of replay material on the 
remote control device. 

Figure 1 is a schematic diagram of a home entertainment system comprising a 
30 number of system components linked by an IEEE 1394 network 10. The components 
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include a satellite dish and receiver 20, a television 30, a computer such as a Personal 
Computer (PC) 40, telephone ^paratus 50, an audio receiver, recorder and playback 
device 60, a video storage device 70, a data storage device 80, a digital camera 90, a 
doorbell monitor 95 and a set top box (STB) 100. Remote control units 110 are 
5 arranged to interact with the STB 1 00. 

In the embodiment of Figure 1, the STB is arranged as a master device to have 
operational control over the other devices attached to the IEEE1394 network 10. So, 
for example, the STB 100 may control the television screen 30 to display pictures 
received by the satellite receiver 20 while the video storage device 70 records pictures 
1 0 captured by the digital camera 90, and so on. The routing of multimedia data between 
the different devices takes place under the protocols established by the IEEE 1394 
standard. 

The doorbell monitor is arranged to monitor not only when the user's doorbell 
rings, but also Aether the house door is then opened. The use for this feature will be 

15 described further below. 

Figure 2 schematically illustrates the set top box 100. Internally, the set top 
box comprises a number of components connected to an internal data and address bus 
arrangement 150. These components include a central processing unit (CPU) 160, 
read only memory (ROM) 170, random access memory (RAM) 180, a real time clock 

20 (RTC) 190, data storage such as a disc drive 200, a remote control interface 210 for 
interfacing to the remote controls 110, a device controller 220 having device drivers 
for the various devices attached to the IEEE 1394 network, a telephone and/or ISDN 
interface 230 for cormecting to a telephone and/or ISDN line 240, and an IEEE 1394 
interface 250 for connecting to the IEEE 1 394 bus 10. 

25 The operations of the STB to be described below take place by suitable data 

and signal processing programmes stored in the ROM 170, the storage device 200 
and/or the RAM 180 being run by the CPU 160. 

Figure 3 is a schematic diagram illustrating the formation of a user action log 
by the STB. • 
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The description of Figure 3 takes the example of television programmes, but it 
is equally applicable to audio (radio) programmes or any other broadcast material 
possibly including terrestrial broadcast, satellite broadcast, cable transmissions, 
material broadcast via the internet and so on. In fact, in this description, "broadcast" 
5 material refers to material for which the user has little or no control over the arrival 
time of the material. 

Referring to Figure 3, for each programme viewed by the user, the STB 
accesses data relating to that programme from an electronic programme guide (EPG). 
EPGs are becoming widely available either as part of the bit stream providing the 
10 actual broadcast material or as separate digital iiiformation from an external EPG 
vendor, for example being accessed via the internet or a dial-up modem coimection. 
An example of part of the information contained in an EPG is as follows: 



Chaimel 


17:00 


17:30 


18:00 


18:30 


BBCl 


Blue Peter 


Neighbours 


National News 


Regional News 


BBC2 


Red Dwarf 


Does he take 
sugar 


The Simpsons 


The Simpsons 


ITV Carlton 


Hollyoaks 


Home and 
Away 


London Tonight 


Sizzled 


ITV Meridian 


HoUyoaks 


Home and 
Away 


Meridian 
Tonight 


Sizzled 


C4 


The Cosby 
Show 


The Cosby 
Show 


The Munsters 


The Munsters 


C5 


Family Affairs 


Family Affairs 


FILM: Hercules 


FILM: 
Hercules 


FOX 


The Racoons 


The Racoons 


Genuine Risk 


Genuine Risk 


CBS 


Up to the 
Minute 


Up to the 
Minute 


CBS News 


CBS News 
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CNN 


Evening 
Edition 


Evening 
Edition 


livening CrQiuon 


Evening 
Edition 


ABC 


It happens 
every „. 


Thursday 


It happens every 


Thursday 


Sky 1 


btar Irek: Dby 


Dtar ireK. Uoy 


Aena 


Aena 


Discovery 


Mother Nature 


Acom: The 
Nature 


Nut 


Jungle Cubs 


TNT 


CHiPs 


CHiPs 


Starsky & Hutch 


Starsky & 
rlutcn 


Nickelodeon 


You're on 
Nick 


Charlie Brown 


You're on Nick, 


Charlie Brown 


Encore 


Just one of the 
Guys 


Just one of the 
Guys 


Just one of the 
Guys 


Just one of the 
Guys 


HBO 


Nothing in 
Common 


Nothing in 
Conunon 


Nothing in 
Common 


Nothing in 
Common 



In addition, the EPG provides, for each listed programme, information such as 
the programme title, a brief description, a programme category, for example selected 
from a standard category list and the people concemed in the making of the 
5 programme, such as actors, directors and so on are listed. If any one of these items is 
missing from the particular EPG entry relating to a current prograiiune, this is not a 
problem for the operations to be described as that item can simply be ignored in the 
following description. 

An example of a part of such a standard category list is as follows. The 
1 0 categories are arranged in a hierarchical stmcture, with a small number of fundamental 
categories, each broken down into sub-categories and so on. As the whole list would 
extend over many pages only a few of the categories are listed here. . 

Advertising: Teleshopping 
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Public service announcements 



Animation: 



Cartoon 

Feature length cartoon 

Plasticine 

Puppets 



10 



15 



25 



Culture : Documentary 
Documentary: 



Event 
Event: 



Discussion 
Discussion: Debate 
20 Debate: 



Architecture 
Cinema 
Fashion 
Jazz 

Classical music 
Jazz 

Literature 
Theatre 



Constmiers / Everyday Life 
Politics 



30 



and so on. 

An alternative standard for programme categorisation has been proposed by the 
DVB consortium. 

At a step 260, so-called key words are parsed from the prograrhme information 
provided by the EPG: The parsing operation is carried out with the assistance of an 
electronic dictionary 270 such as the so-called Wordnet dictionary distributed by 
Princeton University and available at the priority date of this application on the 
internet at http://www.cogsci.princeton.edu/-'Wn/ 
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The Wordnet programme and associated dictionary is capable of establishing 
which words within the programme description are nouns. These words can then be 
passed to subsequent steps for addition to the user action log. 

The step 260 also involves a further search for actors' names by comparison 
5 with a database of actors' names stored in the dictionary 270. Again at the priority 
date of this application, a list of over 6000 television and film actors' and actresses' 
names can be found on the internet at: 
http://dir.yahoo.com/Entertainment/Actors_and_Actresses/ 

At a step 270 a test is formed as to whether the user watched at least half of the 
10 current programme. This test is carried out by establishing the start and stop times of 
the user's viewing (with the aid of the real time clock 190) and comparing these to the 
duration of the programme as specified in the received EPG. If less than half of the 
programme was viewed, details of that programme are not added to the user action log 
and control returns to the step 260 to be repeated for the next-viewed programme. 
1 5 However, if at least half of the programme has been viewed, control passes to a step 
280 where an entry is made relating to the current viewing activity in the user action 
log 290. 

The user action log 290 is effectively a list of actions and associated 
fi-equencies of occurrence, maintained over a predetermined number of actions such as 

20 the most recent 1 000 actions. 

Each action relates to a particular viewing operation. When an action (i.e. 
viewing the current programme) is added to the user action log, the frequency of 
occurrence for each of the key words identified in the step 260 is incremented. (If a 
key word is not present in the log at all, that key word is added with an initial 

25 frequency of occurrence of 1 .) Also, the fi-equency of occurrence is incremented for 
data relating to the particular period of viewing the channel carrying the current 
programme. So, for example, if the user has viewed a programme on channel x 
between 7pm and 7.30pm on a Tuesday using a television set located in the user's 
kitchen, the occurrence frequencies for the following entry in the user action log is 

30 incremented: 
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Tuesday, 7-7.30, channel X, kitchen TV 

In this way, over the course of time, a user action log is built up giving a 
history of that user's viewing habits and preferences. Parts of the user action log will 
be shown graphically below, but the user action log can also be considered as a table of 
5 events and associated frequencies of occurrence. A schematic example is shown 
below: 



Event Type 


Details 


Nvnnber of Occurrences 


Programme category 


News 


37 




Animation: puppets 


11 




Entertainment: soap opera 


259 




Discussion: debate: politics 


2 




• • • 




Programme Director 


Sydney Lumet 


13 




Stephen Spielberg 


6 




Grant / Naylor 


3 








Viewing time 


TV: 19:00- 19:30 


135 




TV: 08:00 - 08:20 


22 




TV: 13:35 - 14:00 


126 









Although the information shown above appears to relate to one user or to a 
1 0 collective analysis of whoever is using the equipment at any time, this is just for clarity 
of the above description. In fact, a user action log is maintained for each individual 
user and, in some embodiments, for each permutation of groups of users of the system. 
Separate logs can also be maintained for different periods^ in the week, e.g. weekends 
and weekdays. 
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In order to do this, the system has to detect vAio is currently using the system at 
any time. In the preferred embodiments, this is carried out as follows. Each user has 
an associated personal remote control device 110. When the user starts to use the 
system, for example to watch or listen to a progranune, the user "logs in" to the system 
5 by pressing a button on that user's remote control device 110. This causes a message 
to be transmitted from that remote control device to the STB, identifying that user and 
indicating that the user has started using the system. When the user is ready to finish 
using the system, the user again presses an appropriate button on his remote control 
sending a corresponding "log out" message to the STB. In this way, the STB 

10 maintains a record of all currently logged-in users, and so adds a new entry relating to 
a currently viewed programme to the user action log appropriate to either the current 
individxial user or to the group of currently viewing users. 

Figure 4 is a schematic diagram illustrating the process by which the STB 
derives an "adaptive progranune guide" and a "virtual channel" of proposed viewing 

15 for the currently logged-in user(s). In the embodiments to be described these are 
derived from the user action logs, but in other embodiments 'v^^ere a "learning" system 
is not employed to generate the user £u:tion logs, a user coiild input manxially a series of 
criteria for use in deriving the virtual channel and adaptive programme guide. These 
criteria would be stored in a file and used in the same way as the user action log in the 

20 description below.! 

One of the disadvantages of broadcast systems providing very many broadcast 
channels is the complexity of the programme guides. With analogue terrestrial 
television and radio, the user had only to consult prograncmie listings for perhaps four 
or five channels. This could easily be done using a printed programme guide. With 

25 systems currently being implemented (including analogue cable and satellite systems 
as well as the newer digital broadcast systems) the number of channels available is 
huge, possibly 200 or more, and so even with an on-screen EPG the user could be 
faced with searching for a long time before being able to select the best viewing 
schedule for an evening's television. 



10 



This disadvantage is addressed by the present embodiments, in that a virtual 
channel and an adaptive programme guide are generated for display to the user. 

The virtual channel is a concatenation of broadcast items from various different 
(real) broadcast channels, put together to form a contiguous or near-contiguous 
5 viewing session (e.g. an evening's viewing). The programmes included in the virtual 
channel are selected by the STB to be those which have the highest objective likeness 
to the viewer's previous viewing habits and preferences. As an example, a virtual 
channel comprises a successive list of programme titles and respective channel 
numbers as follows: 

10 



Time Period 


Channel 


Programme Title 


17:00-17:30 


BBC 2 


Red Dwarf 


17:30-18:00 


BBC 1 


Neighbours 


18:00-18:30 


CNN 


Evening Edition 




• • • 


• • • 



To watch the virtual chaimel, the user simply accepts the recommendations 
made by the STB. The STB maintains the consecutive list in memory as shown above, 
and, by reference to the real time clock and/or programme delivery control (PDC) 

15 information transmitted along with the broadcast progranmies, switches between 
channels at the appropriate times. 

The adaptive program guide is a similar list of broadcast items of at least a 
predetermined level of interest assessed objectively by the STB using the viewing 
nabits and preferences stored in the \iser action log. So, the virtual channel provides 

20 those broadcast programmes having the highest objectively determined likeness to the 
viewing habits and preferences stored in the user action log, so that the virtual channel 
proposes only one programme at any one time. In contrast, the adaptive programme 
guide can have any number of progranmies for a single time slot, subject to those 
programmes having at least a minimum likeness. The adaptive programme guide 

25 therefore removes programmes assessed not to be of interest from the list presented to 
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the user. Because the adaptive programme guide includes prograiiunes other than the 
most likely selection, it is effectively a superset of the virtual channel, including all of 
the programmes of the virtual channel and possibly many more. The user can select 
programmes from the adaptive programme guide to watch instead of those proposed in 
5 the virtual channel, for example by editing the virtual chaimel to replace a programme 
originally selected by the STB for inclusion in the virtual channel with another 
programme from the adaptive programme guide. 

So, Figure 4 schematically illustrates the selection made by the STB of those 
programmes to be included in the virtual channel and the adaptive programme guide. 

10 The selection can be made at any time after the EPG information has been received for 
a particular day, though in the present embodiment the time at which the STB executes 
the steps of Figure 4 and selects programmes for the virtual channel and the adaptive 
programme guide for a particular day is at 1 1pm the evening before that day. 

The process shown in Figure 4 refers to three main inputs: the EPG, the user 

15 action log 290 and a table of user privileges 300. User privileges, and particularly their 
use when groups of users are viewing, will be discussed below in more detail with 
reference to Figures 10 to 12. However, for the purposes of Figure 4, it is sufficient to 
consider that the user privileges define types of programmes which the users are or are 
not allowed to view, and are set by a system "superuser", such as a parent. So, for 

20 example, a yoxmg child might be barred by the system from watching any broadcast or 
pre-recorded programme having an adult content, expressed as an adult certification. 
The effect of this on the operation of Figure 4 is that progranmies v/bxch do not comply 
with the user's privileges will not be included in the virtual chaimel or adaptive 
programme guide for that user. 

25 At a step 3 10, the next date and time range to be considered is set This might, 

for example, start at midnight and increment through the day in ten minute intervals. 
At a step 320 a next channel is selected, and the programme information for that time 
period and that channel examined. At a step 330 the selected progranune is compared 
with user privilege data 300. If the programme does not comply with the user's 

30 privileges then control is passed to a step 340 where a test is carried out to detect 
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whether all channels have been tested for the current date and time range. If this is not 
the case, control returns to step 320 where the channel is incremented. 

If, at step 340, all available channels have been tested for the current date and 
time range, control passes to a step 350 where a test is carried out to detect M^ether all 

5 date and time ranges in the current period of mterest (e.g. a day) have been tested. If 
so, then the whole process ends. If not, then control passes back to step 310 v/hcK the 
date and time range is incremented. 

Returning to step 330, if the current programme does comply with the user's 
privileges, the "likeness" between that programme and the user action log for that user 

10 is assessed at a step 360. This assessment will be described in more detail below with 
reference to Figures 5 to 8, but in brief, a likeness value is generated having a higher 
value for better degrees of objectively assessed likeness between the user action log 
and the current programme. 

At a step 370 the likeness value is compared with a threshold. If it is greater 

15 than the threshold then the programme is passed for inclusion in at least the adaptive 
prograirmie guide, and possibly in the virtual chaimeL So, at a step 380 the likeness 
value for the current programme is compared with the likeness value for the 
programme currently in the virtual charmel for that time period. If the current 
progranraie has the higher likeness value, it replaces the programme in the virtual 

20 channel which is in turn demoted to the adaptive programme guide. Otherwise, the 
current programme is placed directly in the adaptive progranune guide. Control passes 
again to the step 340, as described above. 

When the virtual charmel and the adaptive program guide are displayed, the 
user might find that there is nothing of interest in the proposed selection. The user can 

25 then adjust the threshold for inclusion of items in the adaptive prograrxmie guide, for 
example by operating a key on the remote control or by moving a screen icon such as a 
slider. Setting the threshold to zero is akin to switching off the adaptive progranmie 
guide function, as the whole electronic programme guide will be displayed. 
Functionality and selection by the adaptive progranune guide is maintained, however, 

30 for non-zero threshold values. 
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Figures 5 to 8 schematically illustrate the derivation of the likeness value 
described above. 

In particular. Figure 5 schematically illustrates the programme infomiation 400 
from the EPG associated with a particular programme. The programme infomiation 
5 includes a sentence describing the programme. This is parsed to fomi a list of 
keywords 410, primarily nouns, for example using the "Wordnet" program and list of 
personnel described above. The keywords are compared with frequencies of 
occurrence of those keywords previoxisly stored in the user action log. Figures 6 to 8 
illustrate the way in which this is done. 

10 Figure 6 schematically illustrates a set of keywords under the grouping 

"Director". Figure 7 illustrates a set xmder the grouping "female star". Figure 8 
illustrates a set under the grouping "category", i.e. a category from the long list of 
programme categories described above. 

In more detail, as a progranmie is watched for at least half its length, details of 

15 the progranmie are stored in the user action log as described above, by incrementing 
the frequency of occurrence for each category, key word, date and time, title etc. In 
the same way, negative attributes can be stored for progranmies which are 
contemporaneous with the watched programme and which are not watched. For 
example, the user action log entry for a particular category etc. can be decreased by, 

20 say, 0.5, for a programme which is not watched while another programme is watched 
for at least 50% of the watched programme's length. 

The characteristics can be stored in the user action log as three ranks or orders. 
For the example of the attributes title, time, day and channel (i.e. leaving aside 
keywords and personnel for the purposes of this table), these would be: 

25 

1st order: title, time, day of the week, charmel 
2nd order: (title & time), (title & day of the week), (title & channel) 
30 3rd order: (title & time & day of the week), (title & time & channel). 
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(time & day of the week & channel) 

In this w^y, the significance of various combinations of programme attributes 

and viewing habits can be detected. 
5 When the likeness value is to be assessed for a new programme, the match 

between features of the new programme and the stored attributes in the user action log. 

In particular, the attributes of the new programme are detected and arranged in the 

three orders described above. These ordered characteristics are then considered as 

characteristic values Cq Cm, where 
10 Ci = S (UAL values) / 2 (actions contributing to log for that characteristic) 

where UAL signifies "user action log". 

A recursive calculation can then be performed to determine the likeness value 

for the new programme. The recursive calculation derives a value I by a series of 

ordered steps, steps i=0 to i=M. 

15 



step 0 


Io = 0 


step 1 


I,=Io + (l-I,)Ci 


• • • 




step i+1 


Ii,,=Ii + (l-Ii)Ci 


• • • 




step M 





The derived value is then considered to be the interest level or likeness value 
for the newly considered programme. 

In other embodiments, the likeness fit can be assessed not only from the 
20 frequency of occurrence associated with each keyword, but also from the (objectively 
assessed) importance which the user is deemed to have attached to that grouping of 
keywords- So, for example, because the grouping under "Director" is a highly peaked 
distribution, it is assumed by the STB logic that the "user attaches a lot of importance to 
the choice of director and so the likeness value will be weighted more towards a good 
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fit with the user's favourite directors. In contrast, the distribution shown in Figure 7 is 
fairly flat, v^ch suggests that the user does not place great weight on the choice of 
female star in a programme. Accordingly the likeness value will be weighted so as to 
have little bearing on the choice of female star. It is noted here that one of the 
groupings used is the date, time and channel of the programme, so that if a user always 
watches at a particular time on a particular channel, that will be reflected in the 
calculation of likeness value of a programme at that time and channel. 

A statistical test which detects the flat or peaked nature of a distribution is the 
so-called "entropy" test. A distribution having a low entropy is a relatively peaked 
distribution, and one having a high entropy is more flat. So, the likeness value is 
weighted generally inversely according to the entropy of each grouping of data. 

In further embodiments, the user action log can be maintained in two temporal 
partitions, one for weekday viewing and one for weekend viewing. Here, the 
definition of weekday and weekend depends on local cultural and religious practices, 
but in the United Kingdom the weekend would generally be considered to be Saturday 
and Sunday, although these days could be altered by a user setting of the STB. 

In other embodiments, the user can control the adaptive programme guide to 
display only those programmes in a certain broad category such as sport, news etc. 
This will lead to the display of programmes within that category but still having at 
least a certain objectively detemiined interest level for the user. 

If the likeness value exceeds a second, higher, threshold then further action can 
be taken as illustrated schematically in Figure 9. In particular, the operations of Figure 
9 ensure that even if a user is absent (not viewing), a programme having a very high 
likeness to that user's established viewing habits and preferences will be automatically 
recorded for viewing later. 

In Figure 9, a programme having a high interest level (a high likeness value) 
for a particular user is about to be broadcast. The programme is within the adaptive 
programme guide, but in order to trigger the process described below a further 
threshold value for the programme's likeness value could be applied. 
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A test is first performed at a step 430 to detect whether that user is currently 
viewing the system (i.e. is the user logged in?). If so, then the process (at least as 
regards that particular programme). 

If the user is not present, then at a step 434 a warning is issued to the user. 
This could be for example a pre-recorded audio announcement being played over the 
loudspeakers 60, a visible warning on the television 30 or computer 40, a message 
paged using the telephone 50 and so on. The skilled man will ^preciate that many 
different types of warning are possible with the collection of interconnected equipment 
of Figure 1. 

After the warning has been issued the system pauses for a time x to allow that 
user to log in to the system if he so desires. The tune x could be, for example, one 
minute. After the pause, a further test is performed at a step 438 to detect whether that 
user is viewing. If he is, then the process ends as regards that programme. (The user 
can be left to decide for himself what to view and/or record at that stage). 

If the user is still not present at the step 438 then at a step 440 the likeness 
values of the programmes of the adaptive programme guide for the current time are 
compared with a second, higher, threshold. It is known that they all exceed a first 
threshold and comply with the user's privileges in order to have entered the adaptive 
programme guide, and it is known that the programme of interest which triggered the 
process is in the adaptive programme guide. The likeness values can preferably be 

stored with the progranune details in the adaptive programme guide, or alternatively 

could be recalculated at this stage. 

If the likeness value of a current programme in the adaptive programme guide 

exceeds the second threshold then it is recorded at a step 450 using the recording 

device 70. An electronic mail message can also be sent to the xiser at a step 460 to 

inform him that the progranune has been stored. 

If more than one programme of the adaptive progranmie guide for the current 

time exceeds the second threshold, then this situation can be handled in various ways. 

If there is the capacity to record only one progranune at a time, then the programme 

with the highest likeness value will be recorded. Since this by definition will be fi-om 
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the virtual channel, the test at step 440 could just have referred to the likeness values 
for the virtual channel progranunes. Alternatively, if there is capacity to store more 
than one programme, those having the highest likeness values are stored. 

Figures 10 to 12 schematically illustrate the way in which the privileges and 
5 preferences for multiple users are combined. In particular. Figure 10 is a schematic 
diagram illustrating user privileges when a single user is operating the system of 
Figure 1, and Figures 11 and 12 illustrate different ways of handling group user 
privileges when two or more users are operating the system. 

Referring to Figure 10, a single user logged in to the system has a user ED 500 
10 which in turn is used to specify a virtual chaimel and adaptive programme guide 510. 
As a result of these the user controls 520 his viewing on a television screen 530. 

When a second user logs in, as shown in Figure 11, the individual adaptive 
programme guides 510 are no longer relevant. Instead, a group adaptive programme 
guide and virtual channel 560 , derived from a user log relating to viewing actions 
15 previously undertaken collectively by that grotip and the lowest common privileges 
associated with the group members is used to control 570 programmes shown on the 
television screen 580. 

In an altemative arrangement, user IDs 590 control the creation of a group 
adaptive programme guide and virtual channel 600 form a sum or other combination of 
20 the user action logs 610, 620 associated individually with each of the two users. This 
in turn controls 630 programmes shown on the television screen 640. 

In either of Figures 1 1 and 12, the privileges and preferences established when 
the fu-st of the users logs in are modified when another user logs in. 

Figure 13 schematically illustrates the derivation of highlight and/or preview 
25 programme clips as performed by the STB, 

A received programme is stored on the video storage device 70. With the 
received progranmie so-called "metadata" (accompanying data) is provided by the 
programme initiator. The metadata can be stored with the programme on the video 
storage device 70 or separately on, say, the data storage device 80. 
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The metadata defines two things in relation to successive time periods (e.g. 
frames or small groups of frames) of the received video signal. Firstly, the metadata 
flags parts of the received programme which could be watched as a user preview in 
order to attract the user to watch the programme but without giving away the story. 

5 Secondly, the metadata identifies parts of the programme giving highlights - for 
example, goals being scored in the case of a football game or major story lines in the 
case of a soap opera. Indeed, the highlight information could be categorised, 
particularly in the case of sports events. For example, in the case of a motor race a first 
category could indicate major overtaking manoeuvres in the race, a second category 

1 0 indicates a major crash and so on, so that a user could select highlights according to his 
taste. 

Accordingly, the user can request either a preview or a set of highlights for a 
stored programme. The programme is replayed along with the accompanying 
meudata. The STB detects and causes to be displayed those parts of the programme 
15 flagged by the metadata as containing highlight or preview material. 

To speed the process, the STB can dump the highlight or preview material to 
an intermediate store so that it can be displayed substantially shamelessly to the user. 

User preferences regarding highlight or preview material for different 
categories of programme can be stored as a user profile, these being compared with 
20 details derived from the EPG. 

In a further embodiment, the apparatus can be arranged to record only highlight 
parts of a programme. This might be appropriate if a user is going to be away from 
home for some time but wishes to see the major developments over that period in, say, 
a favourite soap opera. 

25 If the user is watching a series of programmes from the virtual channel (or a 

series otherwise selected) it may be that a gap occurs between desired programmes. 
The STB is operable to propose material to be displayed during this time gap. 
- On detecting a time gap in the virtual channel, the STB seeks material from 
other sources to fill the gap by trying to obtain material corresponding in some way to 

30 the material to be viewed either immediately before or immediately after the gap. 
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A first step in doing this is to establish a set of key words, as described above, 
for the prograntune to one side of the time gap. These key words are then used by the 
STB as search terms amongst (for example) the following material sources: 

the internet 

stored video material, for example on the video storage device 70 

stored audio material 

image stored on the computer 40 
and so on. These searches can be conducted in advance of the time of the gap. The 
STB then presents the user, via the television 30, with a selectable menu from which 
material to fill the gap (if any) can be selected. 

Sometimes the converse can occur, in that the user is temporarily distracted 
from watching or listening to a programme and can miss a few minutes from within 
the programme. This can seriously affect viewer enjoyment 

In an attempt to alleviate this problem, the STB is operable to buffer the missed 
section using a multisession recording/replay device such as the video storage device 
70 or a random access memory device. 

The start of an interruption is detected in one of at least three ways: firstly, the 
telephone ringing and being answered; secondly the doorbell going and being 
answered; and thirdly by a simple user request via the xiser's remote control device. 

In response to the start of an interruption, the current programme is stored to 
the multisession recording device. At the end of the interruption, preferably signalled 
by the user with the user's remote control device, replay is conMnenced from the start 
of the recorded section. However, the programme has to carry on being recorded and 
replayed a certain time later in order that the user can watch the whole of the remainder 
of the progranmie. This recording and replay mechanism for efifectively time-shifting 
a programme to a few minutes (or whatever period) later may have to cany on 
throughout a contiguous viewing period. 

Figure 14 is a schematic diagram of a remote control device 1 10. The device 
comprises a keyboard 700, a processor 710, a display screen 720 and a bi-directional 
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communications interface 730, all within a housing 740 intended to be carried in the 
user's hand. 

The bi-directional communications interface 730 is arranged to communicate 
data wirelessly with the remote control interface 210 of the STB 100. Since there may 
5 be many remote controls 1 10 operating with the system, the commxmication between 
the remote controls and the STB may operate on different respective carrier 
frequencies (for radio or infra-red commimication) or by some other multiplexing 
scheme such as time division or code division multiplexing. 

The keyboard accepts user input including, amongst other things, login and 
10 logout information so that a user can log in to the overall system (this information 
being transmitted directly to the STB 100), programme selections when that user has 
control of the operation of the system, device controls, for example to start a video 
replay from the storage device 70, and so on. Apart from the login/logout, these are 
generally standard functions of a remote controller. 
1 5 Another technique for logging in to the system might just be to operate any key 

on the remote control 110. If each user has an individual remote control, and signals 
form the individual remote controls are distinguishable by the multiplexing scheme 
used, the simple operation of the remote control would signify the presence of that 
user. A further technique is for the remote control or the STB to include a microphone 
20 and voice recognition software for identifying the user. 

The screen 720, for example a colour LCD screen of the general type found in 
palmtop computers, can display information originating from the remote control or 
from the STB. As examples of information originating from the remote control, when 
a device instruction key is operated by the user, there might be various options relating 
25 to that device operation - for example, the speed to be used in a shutUe replay 
operation. These options can be presented to the user in the form of a menu or an 
adjustable screen icon generated by the processor 710; so that the user can select the 
required option to be transmitted to the STB. 
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The possibilities for display of information originating from the STB, or from 
the home entertainment system in general, are very large. Some examples will now be 
described. 

(i) Personal Content 

A user appointment diaiy and address/telephone book can be maintained on the 
data storage device 80. The user can therefore access this type of data for display on 
the screen 720 by commimication with the STB. The way in vAnch the data is stored 
and displayed can be otherwise conventional, the difference being that the storage is 
performed centrally rather than locally at the remote control. 

(ii) Internet browsing 

The user can browse the internet as though the remote control was a terminal 
directly connected via a modem link to an internet point of presence. Again, the 
browsing techniques can be otherwise conventional, but the modem and the software 
can be held centrally and operated under the control of the STB, with the remotely 
linked remote control being a wireless terminal. 

(iii) Programme Guides 

The user can request display on the screen 720 of a programme guide - perhaps 
the full EPG, the adaptive programme guide described earlier or the virtual channel 
also described earlier. These can be transmitted to the remote control from the STB 
via the conununications link. The user can then page up and down the programme 
guide and select one or more programmes for reception and/or recording. The details 
of the selected programmes are then retransmitted back to the STB and the requests 
actioned. 

(iv) Related Content 

The user, viliile watching or listening to a particular broadcast or recorded 
programme, can request so-called related content to be displayed, on the screen 720. 
Related content might include teletext information, subtitling, intemet or recorded 
material derived in the same way as described above for the "gap filling" process, and 
so on. In this way, the user can view related material on the screen 720 without 
disrupting viewing on the main screen 30 for other xisers. 
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The process for requesting related content is illustrated schematically in Figure 
15. Figure 15 is a schematic flow chart divided into two columns. On the left-hand 
column operations by the "fixed" system, in particular by the STB 100 and the 
television 30, are illustrated. On the right hand column are Ulustrated operations 
5 carried out by the remote control 110. 

A step 800 represents the viewing of a programme on the screen 30. As 
represented by the line 810, the viewing on the main screen 30 continues uninterrupted 
by the remainder of the processing to be described. 

At a step 820 the user requests additional content by pressing an appropriate 
10 key or selecting from an appropriate menu on the remote control 1 10, This is at this 
stage a non-specific request. The remote control 1 10 does not have any local record of 
what additional content might be available, so the request is to provide the user with 
information as to what additional content might be available. 

At a step 830 the STB initiates a check of what relevant additional content is 
15 available. This is carried out using the processes described above, whereby terms 
derived from the EPG entry relating to the cunrenUy viewed programme are used as 
search terms to search the internet, other programmes in the EPG, lists of pre-recorded 
material and so on. At a step 840 a list of available material is prepared and 
transmitted to the remote control 110. If the list has a large number of entries, the 
20 most likely n entries, where n is a suitable number to be displayed clearly on the screen 
720, can be transmitted. 

At a step 850 the user selects one item from the list of available material, and 
the selection is sent to the STB. At a step 860 the STB initiates retrieval and sending 
of the selected material to the remote control 110, and finally at a step 870 the 
25 requested material is displayed on the screen 720 and the process ends. 

(v) Replays 

Figure 16 schematically illustrates a similar -process whereby the user can 
request and view on the screen 720 a replay of, for example, the last few seconds of 
material viewed on the main screen 30. witiiout disrupting fijrther viewing on the main 
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screen 30, This can be useful, for example, in sports matches where one user might 
wish to see a replay of a goal being scored. 

Again, Figure 16 is divided into the same two columns, and again it starts with 
a step 900 representing viewing of the main programme, which continues 
5 unintemipted throughout the process of Figure 1 6. 

At the same time as the programme is being viewed, the STB controls the 
temporary storage of a most recent y seconds - for example 60 seconds - of the 
programme to a multisession recording device such as the data storage device 80. So, 
at any time, the device 80 holds at least (see below) the most recent 60 seconds of 
10 viewed material! 

At a step 910 the user requests a replay by pressing an appropriate button on 
the keyboard of the remote control. (This could be done by a menu entry but since the 
user may suddenly wish to do this, it is preferable to have a one-key entry for this 
function). 

15 At a step 920 the STB locks (temporarily prevents deletion of) the currently 

held buffer file of the 60 seconds preceding the issue of the replay command and 
continues the recording of a further most recent 60 seconds of material. The newly 
recorded material is needed in case another user issues a replay command separate to 
the one already being processed. 

20 At a step 930 the replay material is retrieved from the storage device and sent 

to the remote control for display 940 on the display screen 720. Initially, a 
predetermined part of the recorded y seconds of material can be displayed - for 
example, the earliest part, but since the user will be interested in a particular part of the 
material, the user is given the opportimity at a step 950 to shuttle backwards or 

25 forwards through the buffered material to search for and view the required section. In 
this step, certain buttons on the keyboard 700 (for example, backwards and forwards 
arrow keys) can be arranged to cause transmission to the STB of forward and 
backward shuttle conunands, affecting the current replay position within the buffered 
material. 
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Steps 930, 940 and 950 continue as a loop until the user has seen all that he 
requires of the replay material. When the user issues a "stop" command, for example 
by another key of the keyboard 700, the process ends 960 and display ceases on the 
display screen 720. In response to this, at a step 970 the STB can now unlock and 
delete the y seconds of material used for the just-finished replay process. 
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CLAIMS 

1 . Television receiving apparatus comprising: 
a signal receiver; 
a system controller; 

a main display screen operable to display a video signal from the signal 
receiver; and 

a remote control operable to commimicate with the system controller, the 
remote control having a display device and a user-operable control; 

in which the remote control comprises means, responsive to user operation of 
the xiser-operable control, for requesting transfer from the system controller of 
information relating to a video signal currently displayed on the main screen, the 
infomiation being communicated from the system controller to the remote control for 
display on the display device of the remote control. 

2. Apparatus according to claim 1 , in which the system controller comprises: 
means for receiving an electronic programme guide providing data defining 

one or more keywords associated with a currently received programme; and 

means for initiating an intemet search using the key words for the current 
programme to derive information relating to the programme for transmission to the 
remote control. 

3. Apparatus according to claim 1, in which the system controller comprises a 
buffer signal store for storing a most recent predetermined period of the cxirrently 
displayed video signal. 

4. Apparatus according to claim 3, in which the system controller is operable, in 
response to a command received from user operation of the remote control, to replay 
and transmit data representing the signal stored in the buffer signal store for display on 
the display device of the remote control. 
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5. Apparatus according to claim 4, in ^^ch the system controller comprises 
means for setting a current replay position of data in the bufifer store in response to 
user operation of the remote control. 

5 

6. Apparatus according to claim 4 or claim 5, in which the system controller 
comprises means for inhibiting deletion of a period of stored video signal from the 
buffer store during operation of the remote control to view that video signal. 

10 7. Apparatus according to any one of the preceding claims, in which the remote 
control is operable to communicate wirelessly with the system controller- 

8. A remote control operable with television receiving apparatus having a signal 
receiver, a system controller and a main display screen operable to display a video 

1 5 signal from the signal receiver, the remote control being operable to communicate with 
the system controller and having a display device and a user-operable control; 

in which the remote control comprises means, responsive to user operation of 
the user-operable control, for requesting transfer from the system controller of 
information relating to a video signal currently displayed on the main screen, the 

20 information being conmiunicated from the system controller to the remote control for 
display on the display device of the remote control. 

9. Television receiving apparatus substantially as hereinbefore described with 
reference to the accompanying drawings. 

.25 . 

10. A remote control substantially as hereinbefore described with reference to the 
accompanying drawings. 



BNSDOCID: <GB 2343073A, 




Application No: GB 9823287.9 Examiner: Andrew Fearnside 

Claims searched: 1-10 Date of search: 17 February 1999 



Patents Act 1977 

Search Report under Section 17 

Databases searched: 

UK Patent Office collections, including GB, EP, WO & US patent specifications, in: 
UK CI (Ed.Q): H4F (FAAN. FAAX) 
Int CI (Ed.6): H04N (5/00, 5/44, 5/445) 
Other: Online databases: WPI, JAPIO, INSPEC 



Docnments considered to be relevant: 



Categoi> 


Identity of document and relevant passage 


Relevant 
to claims 


X:Y 


EP 0645927 Al 


(THOMSON) See abstract. 




X: 1.7,8 
Y:3 


X:Y 


WO 98/16062 


(CHANG) See whole doc. 




X: 1.7,8 
Y: 3 


Y 


US 5438423 


(TEKTRONIX) See whole doc. 




Y: 3.4.5.6 


Y 


US 5701383 


(GEMSTAR) See whole doc. 




Y: 3.4.5.6 


X:Y 


Patent Abstracts of Japan: JP10174176 (Matsushita), 26.6.98. 
abstract. 


See 


X:1.8 
Y: 3,4.5.6 


X:Y 


Patent Abstracts of Japan: JP9298677 (Matsushita), 18.11.97. 
abstract. 


See 


X: 1.7.8 
Y: 3,4.5.6 


X 


Patent Abstracts of Japan: JP9233567 (Sharp), 5.9.97. See abstract. 


1.7.8 


X:Y 


Patent Abstracts of Japan: JP8251671 (Denshi), 27.9.96. See abstract. 


X: 1.7,8 
Y: 3.4.5,6 



X Document indicanng lack of novdty or invective tttp A 
Y Document indicating lack of inventive step if combined P 
witb one or more odter documents of same category. 

E 

& Member of the same patent famil y 



DoQimeot inrfkaTtng fnrhnological bac3^ground and/or state of the an. 
Dootment pub l i s he d on or after the declared priority date but before 
the filing date of diis invention. 

Patent do ai m rm published on or after, but with priority date earlier 
than, die fPing date of this application. 



An Executive Agency of the Department of Trade and Industry 

.2343073A_L> 



THIS PAGE BUNK (usfto) 



